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Abstract 

This paper explores the unique curriculum design issues involved with the integration of qualitative data analysis software 
(QDAS) in online instruction of qualitative research. The Qualitative Research III - Analysis course was designed for 
graduate students and offered in a blended online format using the NVivo® software program and the learning management 
system, Desire2Learn. Although embedding the NVivo® software into the fabric of the course afforded several advantages, 
it also posed challenges for the students and the instructor. 

INTRODUCTION 

Qualitative Researeh III is a eourse that provides doetoral students with advaneed instmetion in qualitative researeh 
analysis. The intent of the eourse is to teaeh student how to apply methodologieal skills to their theoretieal and 
philosophieal orientations related to doetoral dissertation researeh. 

This paper explores the unique eurrieulum design issues involved with the integration of qualitative data analysis software 
(QDAS) in online instmetion of qualitative researeh. Course design and implementation issues were eategorized into three 
main themes: student ehallenges, eontent ehallenges, and teehnologieal ehallenges. 

CONCEPTUAL FRAMEWORK OF THE COURSE 

Course delivery 

Qualitative Researeh III - Analysis was developed and taught for the first time in the spring semester of 2004 for doetoral 
students. Prior to the semester, graduate degree programs only offered a single eourse in qualitative researeh methods. In 
response to demand for more advaneed qualitative eourse offerings, a sequenee of three graduate-level qualitative researeh 
eourses were developed and submitted for university eurrieulum review and approval during the spring of 2003. 

• Qualitative Researeh I - Methods 

• Qualitative Researeh II - Theory 

• Qualitative Researeh III - Analysis 

The author was seleeted for the 2003 Summer Program for Enhaneing Teaehing and Learning with Teehnology. This 
professional development university grant award provided training in a newly adopted learning management systems (LMS) 
named Desire2Leam (D2L). Currieulum development efforts for Qualitative Researeh III - Analysis began in earnest 
during the summer training program and extended throughout the fall. 

The eourse was offered in two seetions. One seetion was offered in a graduate eomputer lab and the other seetion was 
offered online. Integration of the two seetions afforded the opportunity to teaeh the eourse in a blended mode. The 
blending eonsisted of traditional elassroom leeture, lab format, and Internet delivery. Eaeh student enrolled in the eourse 
eould aeeess the eourse materials by entering a password on the LMS log-in page. All eontent eould be entered by the 
instmetor and viewed by the students and instmetor using a Web browser. 

Students in either seetion were given the option throughout the eourse to attend the evening elass or partieipate via the 
online offering. This flexibility provided online students who lived within driving distanee of eampus the option of 
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participating in traditional instruction on an as needed basis. By providing all course content online, students that enrolled 
in the weekly graduate computer lab section but were unable to attend a session could access course materials at their 
convenience, thus avoiding potential schedule conflicts. All students were expected to participate weekly in online 
discussion forums and submit assignments electronically. 

Teaching Assistants 

To support the delivery of all three qualitative research courses a small team of advanced doctoral students were selected as 
teaching assistants. Teaching assistants were expected to commit for all three semesters and assume responsibility for 
mentoring four to six graduate students. An ideal arrangement was envisioned whereby the teaching assistant would work 
with the same student group throughout the sequence. The intent of maintaining group membership was to allow 
relationships to build and to enhance group stability (Moallem, 2003). Three of the five original teaching assistants did 
complete the sequence but many of the doctoral students did not enroll in all three courses. This did not pose a significant 
problem since the majority of the student dissertation research studies were not clearly focused at the beginning of the third 
course and therefore the students were not positioned to maximize the learning value of a concentrated three course 
sequence. 

The teaching assistant mentor role, however, was of significant value to enhancing the student learning experience. 
Teaching assistants were able to moderate group discussions and provide students with support and cognitive scaffolding as 
needed. On a support level, the teaching assistants encouraged students and fielded their questions. To enhance learning, 
the teaching assistants primarily functioned at the coaching level of Brescia’s telementoring taxonomy (2003). Each week 
the teaching assistants completed the same course assignments as the other students. In this manner the teaching assistants 
provided a model for the depth of engagement that was expected of the students. The teaching assistants also prompted 
students to expand on points raised in the discussion forums. 

The teaching assistants also provided an invaluable role as liaisons between the students and the instructor. Each week on 
the evening prior to the course meetings, the instructor held seminar meetings with the teaching assistants. Teaching 
assistants were enrolled in a Teaching in Higher Education Seminar. A main theme of these seminar meetings was to 
review proposed course content for the upcoming week and acquire course feedback directly from the teaching assistants 
and indirectly from the students. These discussions provided the context to explore a wide range of issues in teaching in 
higher education. 

Course Communication 

Due to the blended course delivery, students could select asynchronous and synchronous modes of communicating with the 
instructor and other students. The primary means of asynchronous course communication were electronic mail and 
asynchronous discussion forums. In some cases students also made telephone contact with the instructor. For students who 
elected not to attend the class sessions, a hybrid form of synchronous communication was made available through the use of 
the online pager tool in D2L. The pager tool, which was similar to an instant message and chat program, allowed students 
to send electronic messages to other users. During class times the teaching assistants visually monitored and responded to 
their onscreen pagers in D2L. The teaching assistant could also ask the instructor questions during the lecture portion of the 
class that had been sent by an online student using the pager tool. In addition, all students could obtain additional online 
assistance using the threaded discussion forum help line. 

Software Availability 

As part of the curriculum on qualitative data analysis, NVivo® software was taught to students. Use of the software allowed 
students to merge data analysis knowledge gained from the lesson with software skills. This integration was consistent with 
the constructivist approach of using practical contexts (Jonassen, Peck, & Wilson, 1999) and eliminated the problem of 
teaching software skills in isolation of practical contexts (Oberlander & Talbert- Johnson, 2004). Prior to the start of the 
semester, the university purchased a site license for the NVivo® software (QSR International, n.d.). To make the NVivo® 
software available to students who preferred to complete the class using solely the online format, the software company, 
QSR International, provided students the option to purchase a four month time limited version at a substantially reduced 
rate. 

Content Delivery 

Consistent with a constructivist framework, students were provided with a course syllabus and a course orientation (Liu, 
2003). The course syllabus was the tool used to communicate the overall course requirements and expectations. A course 
orientation document provided reference links that helped guide students through the format of the online course. The 
orientation served as an advance organizer to inform students of how the course content would be presented. 



Page 2 of 9 




information, and (b) lesson-specific 
welcome letter. The welcome letter 
blended online format of the course, 
in Figure 1. 

Course Content 

Course Information 

1. Course Syllabus 

2. Course Syllabus (RTFl 

3. Course Orientation (RTFl 

4. Welcome Letter 
Weekly Content 

1. January 13 

2. January 20 

3. January 27 

4. February 3 

Figure 1 : Presentation of course content in the LMS 



There were two mam types of course content provided for students: (a) course-level 
content. The course-level information included the syllabus, course orientation, and a 
had been distributed to students prior to the start of the course to introduce them to the 
Students accessed course content by clicking on the corresponding hyperlink, as shown 



The content of each weekly instruction was presented as a Web page that students could read onscreen, download, or print. 
Within each weekly lesson, content was divided into eight sections: (a) multimedia content, (b) a list of topics covered in 
the lesson, (c) a list of learning outcomes for the lesson, and (d) a list of hyperlinks to the main lesson content, (e) a list of 
required readings for the week, (f) the lesson content and description of assignments, (g) recapitulation of the prior week, 
and (h) an assignment sumtuary. The selection of these headings was based on Gagne’s events of instruction (Smith & 
Ragan, 1999) and the experiences of the instructor’s graduate assistant in supporting online courses. 

Student Interaction 

Student interaction in the course was considered a vital element to enhance the multiple perspectives of qualitative research 
and to reap the benefits of social learning (Moallem, 2003). To facilitate student interaction, students were required to 
submit responses to questions posed by the instructor in discussion threads. The use of asynchronous discussion threads 
was adopted to simulate the types of discussion that would occur in a classroom environment and to allow students to apply 
knowledge gained in the weekly lessons. This use of discussion threads in online learning had been shown to curb feelings 
of isolation (Flarasim, 1987; Poole, 2000) and increase collaboration (Flarasim, 1987; Moallem, 2003). In addition to 
facilitating student interaction, discussion threads had also enhanced student learning experiences in online action research 
courses by providing students with a means of applying content-related knowledge (Liu, 2003). Furthermore, the use of 
asynchronous threaded discussions had been shown to increase the reflexivity and depth of student responses because 
students had more time to simulate their thoughts (Christopher, Thomas, & Tallent-Runnels, 2003). Another reported 
outcome of the use of online discussions was improved critical thinking when students were given learning guidance and 
expectations for providing constructive criticism (Astleitner, 2002). 

The three types of discussion forums employed in the course were open forum discussions, lesson-specific discussions, and 
teaching assistant foruius. The open foruiu discussions, as shown in Figure 2, provided students with a vehicle for posting 
comments, questions, and concerns about general issues in the course. For example, students could share their best 
practices for coping with technology issue in the discussion titled “Lessons Learned - Dealing with Technology”. 

Open Forum Discussions e 

This is an open forum for everybody to share tips and strategies or ask questions and seek assistance. 

• Lessons Learned - C'e.3lino with Technoloov (3S msgs - 0 unread) 

Throughout this course, each of us will be developing our own techniques working with technology. Please use this forum to 
share your tips and strategies with technology. You may just save a fellow student the pain of making the same mistake. 

• Help Line (10 msgs - 6 unread) 

Use this topic if you are seeking or sharing assistance related to course lessons and assignments. 

• Lessons Learned - Working with NVivo (23 msgs - 20 unread) 

• Multiple Meanings - Image Analysis Exercise (27 msgs - 27 unread) 

Figure 2: Open forum discussion in the LMS 
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The instructor, teaching assistants, and students provided support for one another by answering questions and sharing their 
experiences in the discussion threads. This feedback also provided valuable information for the instmctor for formative 
evaluation and revision of the course. 

In addition to the general discussions, there were also discussions that were directly related to weekly lessons. The purpose 
of the lesson-specific discussions was to provide students with a means of applying knowledge gained in the weekly lesson, 
and for providing students with constructive feedback from their peers. 

Consistent with the practices of Moallem (2003) in maximizing group collaboration, students were assigned to groups of 
four to six students. Access to each lesson-specific discussion was restricted based on group membership. Students were 
permitted to view and respond to postings made only by other members of their group. The size of the groups was based 
primarily on the number of teaching assistants enrolled in the course, but was relatively consistent with the recommended 
practice of using groups of three or four students. As indicated by Moallem, the use of small groups, which remained intact 
for the duration of the course, provided stability in group interaction. Restricting discussion forum access by group 
membership also served to reduce the number of postings to which students were required to respond. 

Interaction between the instmctor and teaching assistants was also considered vital to the success of the course. To provide 
the instmctor and teaching assistants with a vehicle for communication, the instmctor created a teaching assistant discussion 
fomm (see Figure 3) that was only available to him and the teaching assistants. These discussions provided a valuable 
exchange of information that lead to formative revision of the course and assignment activities. 

Teaching Assistant Forum e 

• Pain and Headaches (9 msgs - 0 unread) 

• Seminar Assignments (17 msgs - 0 unread) 

• Monday Meeting Schedule (1 msgs - 0 unread) 

• Participation Grade (5 msgs - 0 unread) 

Figure 3: Teaching assistant discussion fomm in the LMS 

DESIGN AND IMPLEMENTATION ISSUES 

Several components of the course framework posed unique challenges to the success of the course. While the instmctor had 
anticipated some of the challenges during the design phase, other challenges arose during the delivery of the course. Course 
design and implementation issues were categorized into three main themes: student challenges, content challenges, and 
technological challenges, as indicated in the model. 

Students 

/ Developing Skills \ 

/ Determining Workload \ 

/ Communicating Requirements \ 



Content 


Technology 


Integrating QDAS 


Saving Project Files \ 


y/ Enhancing Refleixivity 


Grading Project Files N. 


/ Negotiating Perspectives 


Working Project Files \ 



Student Challenges 

Several of the challenges to successful delivery of the course were either imposed by students or directly related to the 
students. These challenges included alignment of students’ actual entry skills with the instmctor’s expectations of students’ 
entry skills. The instmctor’s expectations for students who enrolled in the course were that the students have developed an 
understanding of their research skills in qualitative methodologies. Qualitative Research III - Analysis was designed for 
graduate students at or nearing the completion of all doctoral course work. The instmctor’s expectation was that students 
would enter this course at the beginning stages of their dissertation research proposal. Most students, however, had not 
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articulated a clear research focus at the beginning of the course. Consequently, those students had to commit additional list 
to assignments related to data collection. 

Communicating Requirements 

One of the challenges of delivering an online course was making sure that instructions, expectations, and requirements were 
clearly conveyed (Aune, 2002). The onus was on students to read and critically analyse each assignment because textual 
information was often the only source of information. Clear communication was essential for presenting information to 
students. Within a traditional classroom, the instructor could often determine whether students understand what is required 
based on looks that belie confusion. An online course delivery platform could delay this immediate feedback because the 
instructor had to wait until students submit completed assignments. 

In this course, clear communication was facilitated by the teaching assistants. The teaching assistants also provided a 
weekly written summary of the class lecture. These TA Notes were posted on D2L the following day after each class 
meeting. This supplement to the course content provided online students with a student perspective to understanding 
weekly topics and assignments. 

Determining Workload 

A consistent research theme of online courses is that students perceive an increased workload or time commitment 
(Harasim, 1987; Liu, 2003). Often the basis for this sentiment is due to the amount of reading inherent with online course 
delivery. In addition to having to read weekly content, students must also read what would have been presented verbally 
during class. The corresponding challenge to the instructor was to balance the weekly content load with the assignment 
load. Six weeks into the semester, one of the teaching assistants voiced her concerns in one of the teaching assistant forums 
that the workload was overwhelming. The instructor replied, 

“You raise a major question, one that I struggle with each week. We need to critically look at the assignment load 
each week in our Monday Seminar. I continue to feel tom on this issue. This course represents one of the last 
courses a doctoral student is taking prior to a solo launch into a qualitative dissertation study. The pressure of 
these weekly assignments is mild compared to the dissertation process. Two of our key goals are to help doctoral 
students improve the overall quality of their qualitative dissertations and to give the students a head start on their 
research.” 

The other teaching assistants joined in the threaded discussion by reflecting upon the demanding duties of a teaching 
assistant and various coping strategies for doctoral students. Since the issue continued to be referenced by the teaching 
assistants and instractor in the weekly teaching seminars for the remainder of the semester, the instructor began to 
proactively address the workload concerns. During the weekly meetings with the teaching assistants, the instructor would 
present to them the proposed course content. The teaching assistants would then preview the content and assess the 
workload prior to the instmctor making the content available to students. The LMS provided a conditional release feature 
which allowed the instmctor to suppress content from the students until a specific date had passed or the instmctor activated 
the content. 

Content Challenges 

One of the challenges faced the instmctor when designing the course was how to present the content and stmcture activities 
in the online environment. To compound the complexity, the LMS had been newly adopted by the University. 
Consequently, there was a significant learning curve for the instmctor. Three key strategies that the instmctor adopted were 
enhancing refiexivity skills through researcher collaboration, capitalizing on the importance of negotiating multiple 
perspectives, and integrating computer-assisted data analysis software into the course. 

Enhancing Refiexivity Skills 

During the weeks where memos were the topic of the lesson, students were assigned to conduct field research related to 
their research interests and submit their memos in the appropriate threaded discussion. Each student was tasked with three 
memo topics: methods memos, researcher reflections memos, and analytic memos. This division of memos was defined for 
the students accordingly: 

• Methods memos can be used as an audit trail of your design. These should include how your design develops 
throughout the process of the research. They are also good for keeping track of what has not been completed - 
research lines of inquiry that have not yet been taken. 

• Analytic memos (Glaser & Strauss, 1967) reflect your chain of reasoning for the research. These address questions 
such as “what am I learning here” and “what meanings am I seeing.” 
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• Researcher reflections memos deal with researcher as instrument issues. While all three memo types are 

reflections, these specifically have to do with the impact of researcher on the site and site upon the researcher. 

These are also a good place to address researcher bias, researcher baggage, and struggles with premature attempts 
at deductive reasoning. 

After submitting their memos to the threaded discussions, students were tasked with reading submissions made by other 
students and providing constmctive feedback. Students submitted feedback by submitting replies to the students’ postings. 
The teaching assistants facilitated the virtual group exchange and provided technical and content assistance as needed. 

Negotiating Multiple Perspectives 

To teach students about the importance of multiple meanings and to capitalize on the importance of negotiating multiple 
perspectives, the instructor devised a Multiple Meanings assignment. The multiple meanings exercise demonstrated the 
instructional advantages of dual activity integration. The instructor posted a photograph of the painting “Kept In” (Gilmore- 
Lehne, n.d.), and asked students to view the photograph (see Figure 4). 




Figure 4: Photograph used in Multiple Meanings assignment 



Prior to reading any other students comments, write your own comments on what this picture means to you. 
Submit your comments to the open fomm discussion no later than Saturday. Beginning Sunday immerse yourself 
in the data by reading all comments, then open code and axial code all comments. Identify the major themes you 
have constructed from this process. Do they differ from your original comments? Briefly discuss your results by 
sharing in the open forum discussion. 

The students were given no additional information about the title, nature, or origin of the photograph. 

Integrating Computer-Assisted Data Analysis 

Four weeks later, students returned to this exercise and used the NVivo® software to analyze the compiled class comments 
and analysis. Searching Multiple Meanings Data Assignment: 

You may recall the Multiple Meanings exercise from the February 24 class and my suggestion that we may be 
returning to this discussion for further data mining. This assignment provides you an opportunity to refine your 
data mining skills and explore issues surrounding intercoder reliability. To dig into this data further we need to 
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really immerse ourselves into the finer meanings within the individual entries on meaning and your subsequent 
group analysis. 

Begin by downloading two rtf files eontaining the Multiple Meanings data (Observation and Analysis). Next, 
ereate a Multiple Meanings NVivo projeet and ereate both doeuments into the projeet. Do not mix this 
assignment with yonr NVivo research project. Title this projeet "Multiple Meanings." Onee you have the 
Multiple Meanings projeet ereated, eode the data, keeping in mind Silverman's diseussion on reflexivity, a realist 
approaeh and narrative analysis. As you build your tree nodes, draw upon the eode work of other students 
embedded in the analysis file. Be aware of how other student's eode work is shaping your eoding. This is a 
signifieant point that Silverman raised in the reading. 

To further refine your eode analysis, use the seareh lunetion tool. Submit your Multiple Meanings NVivo projeet 
as a qbk file to the Searching Multiple Meanings Data dropbox. This assignment will involve ereate projeet, ereate 
doeuments, eoding, seareh and baekup. 

This assignment allowed the instmetor to add an essential dimension to the learning environment. The integration of the 
NVivo® software teehnology and virtual exehange into the assignment ereated a synergism between learning eomputer- 
assisted data analysis and exploring negotiating multiple meanings. The value of this synergism was realized from a 
qualitative researeh perspeetive and a learning proeess perspeetive. From the learning perspeetive, the NVivo® software 
allowed students to eonstruet qualitative researeh knowledge through the data analysis exereise while embedding 
knowledge of the analysis tool into the aetivity. Consistent with the eonstmetivist framework, knowledge was both 
embedded in the aetivity and eonstrueted from the aetivity (Jonassen, Peek, & Wilson, 1999). 

Technological challenges 

Although embedding the NVivo® software into the fabrie of the eourse afforded several advantages, it also posed ehallenges 
for the students and the instmetor. Most of these ehallenges were imposed by the funetionality of the NVivo® software. 
The initial ehallenge related to the use of the software oeeurred when students were first required to submit projeet files to 
the instmetor for grading. To submit assignments for grading, students were required to ereate baekup files of their NVivo® 
projeet file. The students were then supposed to upload the resultant file using the dropbox tool in the LMS. During this 
assignment, several students had diffieulty saving the file in the eorreet format. The diffieulties of one partieular student led 
to the instmetor having to reinstall the NVivo® software on his eomputer. 

Another diffieulty experieneed by the instmetor oeeurred as he opened and elosed students’ projeet files for grading. Due to 
the design of the NVivo® software, the instmetor was foreed to rename student projeets as he opened and saved them. 
Although the opening and elosing was not speeifieally the problem, the software left a footprint of the projeet even after the 
projeet file had been deleted from the eomputer’ s hard drive. Consequently, the list of projeet files on the dialog box used 
to aeeess the files beeame tremendously cluttered. Students experienced a similar problem when they created project files 
to submit for grading. The design of the software did not allow students to save new copies of project files with the same 
name as the original file. The design of the program was to append a numeric suffix to the file name. Some of the project 
file names (as shown in Figure 5) became rather long and cumbersome. 



Feedback Left Aor 25. 2004 6;55 PM 
Read: Apr 26, 2004 10:32 AM 


r 


IS 


P^WRlfare Reform 3 2 2 2 2 2 2 Backuo.abk 


Submitted: 

Apr 13, 2004 5:20 
PM 


(288k) 

Comments: 


r 


a 


^ ^Sudenit Name - Welfare Reform 3 2 2 


Submitted: 

Apr 29, 2004 1:26 
PM 


2 2 Backuo.abk (210k) 
Comments: 

Program Model 


r 


a 


^Student Name - Welfare Reform 3 2 2 


Submitted: 

Apr 29, 2004 1:26 
PM 


2 2 BackuotlTabk (210kl 

Comments: 



Figure 5: Sample of project file names 



Once students had mastered the concept of creating backup projects to work from, the next challenge occurred when some 
students created different projects for each weekly assignment. Optimally, students were supposed to build upon the work 
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of one project file and overwrite or otherwise replace their working version of the project file. Some students, though, 
began working with superseded versions of the project file, and were faced with additional tasks to integrate work done in 
both project files. 

One particularly frustrating experience for one of the students occurred when she determined that she had two separate 
project files that needed to be merged. Although the NVivo® software did not include a utility for merging project files, the 
instructor purchased a copy of Merge® software which was designed to perform this task. During attempts to merge sample 
projects, though, the instructor learned that a limitation of the Merge® software was that it did not allow for merging of 
research models created in NVivo® projects. Due to this discovery, the student was forced to recreate all of the work that 
she had initially done in creating a model of her data analysis. 

INSTRUCTOR AND STUDENT EVALUATION 

At the end of the semester, the instructor presented the students with a standardized university course evaluation. The 
instructor then reviewed feedback from the completed evaluations to assist in revising the course for the next offering. The 
evaluation instrument consisted of 18 statements pertaining to the instructor and course, which were rated using a Likert 
scale. Numeric scores and written comments were consistently very positive toward the instructor and course. On nine of 
the items, the instructor and course received the maximum positive ratings of excellent or very good. On the remaining nine 
items the instructor and course achieved 80% of the maximum rating. Students’ comments emphasized that the course was 
extremely beneficial and thought-provoking. On the other hand, students also expressed concern with the challenges of the 
course content. As one student stated, “The course was very labor intensive, integrating theory and two software programs 
presented demanding challenges.” 

Future Directious 

Due to the generally overall positive feedback from the students, the instructor plans to offer the course again using the 
same blended format. Based on the instructor’s experiences and the feedback from the students and teaching assistants, 
though, the instructor plans to make several modifications to improve the instructional delivery of the course. For example, 
more clarity will be provided within the first few class sessions regarding saving NVivo® projects and submission 
procedures to the D2L dropbox. Issues related to the alignment of the course within the three course qualitative research 
sequence are also being reviewed. Attention will be given to better prepare students in the first two courses of the sequence 
concerning dissertation research focus. 

An additional curriculum development benefit has occurred with successful elements from this initiative being adopted for 
the Qualitative Research I - Methods course. The blended delivery format using online and traditional classroom will be 
implemented for the spring, 2005 semester. Teaching assistants will provide mentoring support and one class session will 
be dedicated to an orientation to qualitative data analysis software using the NVivo® program. The asynchronous and 
synchronous communication techniques will be incorporated as well as the continued development of creative dual activity 
learning exercises. 

Suggestions for Curriculum Development 

While varying course structures may pose unique challenges, the challenges presented in this paper provide insight into the 
types of issues that educators may face when implementing an online qualitative research course. In a broader sense, 
curriculum development efforts must become increasingly attentive to the rapidly evolving relationship of software program 
integration with course content. The union of students, content, and technology can lead to a rich learning experience when 
these issues are anticipated and addressed prior to course delivery. 
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